<html>
<head><meta charset="utf-8"><title>Nearly identical Crater runs processed a crate differently · t-infra · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/index.html">t-infra</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Nearly.20identical.20Crater.20runs.20processed.20a.20crate.20differently.html">Nearly identical Crater runs processed a crate differently</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="230406661"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Nearly%20identical%20Crater%20runs%20processed%20a%20crate%20differently/near/230406661" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Nearly.20identical.20Crater.20runs.20processed.20a.20crate.20differently.html#230406661">(Mar 15 2021 at 19:47)</a>:</h4>
<p>I'm looking at the logs of two older Crater runs:</p>
<ul>
<li><a href="https://crater-reports.s3.amazonaws.com/pr-73345-5/try%23ad74635edcad306163ced4c2ca1101d7ff073a38/reg/a2-0.5.3/log.txt">https://crater-reports.s3.amazonaws.com/pr-73345-5/try%23ad74635edcad306163ced4c2ca1101d7ff073a38/reg/a2-0.5.3/log.txt</a></li>
<li><a href="https://crater-reports.s3.amazonaws.com/pr-73345-3/try%23fcacb4957f60720c2fba7d94d96d968d72f18c49/reg/a2-0.5.3/log.txt">https://crater-reports.s3.amazonaws.com/pr-73345-3/try%23fcacb4957f60720c2fba7d94d96d968d72f18c49/reg/a2-0.5.3/log.txt</a></li>
</ul>
<p>These logs are from two different runs, each processing the crate <code>a2 v0.5.3</code><br>
Both runs also appear to have used the same Docker image (<code>rustops/crates-build-env@sha256:819119df93c0f5eca3f29186f14981ef29945a311854d7222af07488600a2584</code>)</p>
<p>However, Cargo appears to have done something completely different in each case. In the run for <code>pr-73345-3</code>, running <code>cargo check</code> appears to have checked the <em>tests</em> as well (note all of the extra dependencies that are getting built for the same crate version). The line is the error message is behind a <code>#[cfg(test)]</code>.</p>
<p>In the run for <code>pr-73345-5</code>, it looks like we didn't check the tests, despite running the same <code>cargo</code> command in the same docker image. This is the behavior I would expect, since I'm pretty sure that <code>cargo check</code> doesn't check tests without the <code>--tests</code> flag. However, I hve no idea why this was different between runs</p>



<a name="230418397"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Nearly%20identical%20Crater%20runs%20processed%20a%20crate%20differently/near/230418397" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lqd <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Nearly.20identical.20Crater.20runs.20processed.20a.20crate.20differently.html#230418397">(Mar 15 2021 at 21:06)</a>:</h4>
<blockquote>
<p><code>cargo check</code> doesn't check tests without the <code>--tests</code> flag</p>
</blockquote>
<p>it does with <code>--all-targets</code> which you can see in these logs near the top, in the command that crater runs</p>



<a name="230436022"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Nearly%20identical%20Crater%20runs%20processed%20a%20crate%20differently/near/230436022" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Nearly.20identical.20Crater.20runs.20processed.20a.20crate.20differently.html#230436022">(Mar 15 2021 at 23:13)</a>:</h4>
<p>Huh. Why did that only work for one of the runs?</p>



<a name="230743167"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Nearly%20identical%20Crater%20runs%20processed%20a%20crate%20differently/near/230743167" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Nearly.20identical.20Crater.20runs.20processed.20a.20crate.20differently.html#230743167">(Mar 17 2021 at 18:29)</a>:</h4>
<p>cc <span class="user-mention" data-user-id="121055">@Pietro Albini</span> - Do you know what might have caused this? I'm concerned that future crater runs will miss regressions because of this</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>